Method and apparatus of extracting text from document image with complex background, computer program and storage medium thereof

ABSTRACT

The present invention discloses an apparatus of extracting text from document image with complex background, a method of extracting text from document image with complex background, computer program and storage medium thereof. The preferred method of extracting text from document image with complex background according to the present invention comprising the steps of: a first edge extracting step of extracting edges which have higher contrast than a first contrast threshold from said image; a searching step of searching connected edges from said extracted edges; a second edge extracting step of extracting edges which have higher contrast than a second contrast threshold in case that the pixels number of said searched connected edges is bigger than a predetermined size, wherein said second contrast threshold is higher than said first contrast threshold.

FIELD OF THE INVENTION

The present invention generally relates to image processing. Inparticular, the present invention relates to an apparatus of extractingtext from document image with complex background, a method of extractingtext from document image with complex background, computer program andstorage medium thereof.

BACKGROUND OF THE INVENTION

Text extraction is a very important step for many applications, such asOptical Character Recognition (OCR), text-based video retrieval,document image compression, etc. Most of current techniques aim toextract text from images with simple background. In recent years, thetechnique of extracting text from complex images is required in more andmore fields, such as complex document analysis, engineer drawingsanalysis, etc. However, it is a very difficult problem to extract textfrom document image with complex background. Many methods have beenproposed by prior researchers, most of them are only effective forsimple or not so complex images.

Current text extraction methods can be classified into two groups:Color-clustering based methods and edge-analysis based methods.

Color-clustering based methods assume that text has homogeneousforeground colors. However, this is not always the case, especially forsmall characters. For example, text characters may be printed withdifferent colors; images may be captured under uneven illuminationconditions. And for small texts, the foreground colors are always notuniform, because the transition region is too wide in comparison withstroke width. Accordingly, it is hard to acquire proper globalbinarizing threshold for the whole image and thus it is impossible toeliminate all the light-colored background with un-uniform colors.

On the other hand, the edge-analysis based methods assume that text hasbig contrast with background. But in images with complex background,non-text objects maybe have big contrast with background, which willcause the text edges and non-text edges touch with each other after edgedetection processing. This often brings difficulty or unstable resultsfor edge analysis.

For example, the Japanese Patent Application Laid-open No.JP-A-2000-20714 has disclosed an image processing method, its device andrecording medium storing image processing function.

FIG. 10 shows the flow chart of the image processing method disclosed bythe above Japanese Patent Application Laid-open No. JP-A-2000-20714.

To obtain a binary image having no noise interrupting recognition evenon a background image, the density image of an original image to bethreshold processed is inputted in step S101 and stored in step S102.Then, in step S103, a certain pixel is noticed and whether the pixel isthe edge of a character or a ruled line or not is judged. Thereafter, instep S104, The pixel value on a binary image of the pixel judged as theedge is determined and stored. These operations are repeated for allpixels on the original image in step S105 and all connection componentsof pixels other than edges are found out in step S106. Then, in stepS107, pixels brought into contact with the periphery of a certainconnection component and having already determined pixel values arenoticed and the numbers of black pixels and white pixels arerespectively counted. The numbers of black and white pixels are mutuallycompared in step S108, and when the number of black pixels is larger,the whole connection component is registered as black pixels in stepS110. In the other case, the whole connection component is registered aswhite pixels in step S109. The operation is repeated for all connectioncomponents in step S111, and finally a binary image is generated in stepS112 and outputted in step S113.

According to the above described method, the long lines formed by theConnected Components appearing in the background can be recognized andremoved from the binarized edge map. However, in the edge map afterbinarizing, closed text row also may form a long Connected Component. Inthis case, it is not easy to separate the text from the closed text rowand the whole closed text row may be deemed as background and be ignoredaccording to the above disclosed method. Whereas the text row is what isdesired and should not be simply removed. Therefore, if the scanneddocument image with complex background is binarized and processedaccording to the above mentioned prior art, useful text may be lost.

SUMMARY OF THE INVENTION

Accordingly, the object of the present invention is to provide anapparatus of extracting text from document image with complexbackground, a method of extracting text from document image with complexbackground, computer program and storage medium thereof, so as toovercome the above mentioned defects in the prior art.

To achieve the above stated objects, according to an aspect of thepresent invention, there is provided a method of extracting text fromdocument image with complex background comprising the steps of: a firstedge extracting step of extracting edges which have higher contrast thana first contrast threshold from said image; a searching step ofsearching connected edges from said extracted edges; a second edgeextracting step of extracting edges which have higher contrast than asecond contrast threshold in case that the pixels number of saidsearched connected edges is bigger than a predetermined size, whereinsaid second contrast threshold is higher than said first contrastthreshold.

To achieve the above stated objects, according to an aspect of thepresent invention, there is provided another method of extracting textfrom document image with complex background comprising the steps of: anadjusting step of adjusting the contrast threshold; a text areadetermining step of determining where is the text area based on saidadjusted contrast threshold; wherein said adjusting step comprises astep of target area determining step of extracting the edges which havehigher contrast than said contrast threshold from the target area,searching the connected edges from said extracted edges, and determiningwhether the area covering said searched connected edges should be a newtarget area; wherein said adjusting step enlarges said contrastthreshold when said determined new target area is bigger than apredetermined size, and finishes adjustment of said contrast thresholdwhen said determined new target area is smaller than or equal to thepredetermined size; and wherein the text area determining stepdetermines that the target area corresponding to said contrast thresholdwhose adjustment is finished should be the text area.

To achieve the above stated objects, according to another aspect of thepresent invention, there is provided an apparatus of extracting textfrom document image with complex background comprising: a first edgeextracting means for extracting edges which have higher contrast than afirst contrast threshold from said image; a searching means forsearching connected edges from said extracted edges; a second edgeextracting means for extracting edges which have higher contrast than asecond contrast threshold in case that the pixels number of saidsearched connected edges is bigger than a predetermined size, whereinsaid second contrast threshold is higher than said first contrastthreshold.

To achieve the above stated objects, according to another aspect of thepresent invention, there is provided another apparatus of extractingtext from document image with complex background comprising: anadjusting means for adjusting the contrast threshold; a text areadetermining means for determining where is the text area based on saidadjusted contrast threshold; wherein said adjusting means comprises atarget area determining means for extracting the edges which have highercontrast than said contrast threshold from the target area, searchingthe connected edges from said extracted edges, and determining whetherthe area covering said searched connected edges should be a new targetarea; wherein said adjusting means enlarges said contrast threshold whensaid determined new target area is bigger than a predetermined size, andfinishes adjustment of said contrast threshold when said determined newtarget area is smaller than or equal to the predetermined size; andwherein the text area determining means determines that the target areacorresponding to said contrast threshold whose adjustment is finishedshould be the text area.

To achieve the above stated objects, according to still another aspectof the present invention, there is provided an apparatus of extractingtext from document image with complex background comprising: an edge mapcalculation unit for calculating the edge map of the document image; along background connected edges remove unit for classifying the edges inthe edge map calculated by the edge map calculation unit into two typesof “positive edge” and “negative edge”, searching the connected edgesformed by edges of the same type, and removing the connected edgesformed by edges of the same type longer than a predetermined threshold;an edge map recalculation unit for searching connected edges formed byedges of both types in the edge map with the long connected edges formedby edges of the same types being removed by the long backgroundconnected edges remove unit, and recalculating local edge map for thebounding box of a connected edge formed by edges of both types largerthan a second predetermined threshold; a text map mark unit forclassifying the connected edges into three types of “normal-text”,“reverse-text” and “background” and generating a mark map, wherein theforeground pixels of “normal-text” connected edges are marked as“normal-text”, the foreground pixels of “reverse-text” connected edgesare marked as “reverse-text”, and the rest pixels are marked as“background”; and a text connected edge search and merge unit forsearching on the mark map generated by the text map mark unit theconnected edges formed by pixels with the same mark and forming theconnected edges into text rows.

Computer program for implementing the above said method of extractingtext from document image with complex background is also provided.

In addition, computer program products in at least one computer-readablemedium comprising the program codes for implementing the above saidmethod of extracting text from document image with complex backgroundare also provided.

Other objects, features and advantages of the present invention will beapparent from the following description when taken in conjunction withthe accompanying drawings, in which like reference characters designatethe same or similar parts throughout the drawings thereof.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated in and constitute apart of the specification, illustrate embodiments of the invention and,together with the description, serve to explain the principles of theinvention. In the drawings:

FIG. 1 is a block diagram of a computer system, which may be used withthe present invention;

FIG. 2 is a flow chart showing the method of extracting text fromdocument image with complex background according to the presentinvention;

FIG. 3 shows an example for edge classification and results of removinglong background edge Connected Components;

FIG. 4 is a flow chart showing the method of recalculating the localedge map with the feedback of Connected Components size according to thepresent invention;

FIG. 5 shows an example and the result of removing the disturbance ofthe background near the text by recalculating the edge map with thefeedback of Connected Components size according to the presentinvention;

FIG. 6 and FIG. 7 show the results of text extraction by using themethod of the present invention;

FIG. 8 shows a typical application of the method of extracting text fromdocument image with complex background according to the presentinvention;

FIG. 9 is a block diagram of the apparatus of extracting text fromdocument image with complex background according to an embodiment of thepresent invention; and

FIG. 10 is a flow chart showing the image processing method according tothe prior art.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

In the following detailed description, numerous specific details are setforth in order to provide a thorough understanding of the invention.However, it will be appreciated by one of ordinary skill in the art thatthe present invention shall not be limited to these specific details.

Example of Computer System

The method of the invention may be implemented in any image processingdevice, for example, a personal computer (PC), a notebook, or asingle-chip microcomputer (SCM) embedded in a camera, a video camera, ascanner, and etc. To a person skilled in the art, it would be easy torealize the method of the invention through software, hardware and/orfirmware. It should be particularly noted that, to implement any step ofthe method or any combination of the steps, or any combination of thecomponents, it is obvious for a person skilled in the art that it may benecessary to use I/O device, memory device, microprocessor such as CPU,and the like. The following descriptions and the method of the presentinvention will not necessarily mention such devices, although they areactually used.

As the image processing device mentioned above, the block diagramillustrated in FIG. 1 shows one example of a typical computer system,which may be used with the present invention. Note that while FIG. 1illustrates various components of a computer system, it is not intendedto represent any particular architecture or manner of interconnectingthe components, as such details are not germane to the presentinvention. It will also be appreciated that network computers and otherdata processing systems, which have fewer components or perhaps morecomponents, may also be used with the present invention.

As shown in FIG. 1, the computer system, which is a form of a dataprocessing system, includes a bus 101 that is coupled to amicroprocessor 102 and a ROM 104 and volatile RAM 105 and a non-volatilememory 106. The microprocessor 102, which may be a Pentiummicroprocessor from Intel Corporation, is coupled to cache memory 103 asshown in the example of FIG. 1. The bus 101 interconnects these variouscomponents together, and also interconnects these components 103, 104,105, and 106 to a display controller and display device 107 and toperipheral devices such as input/output (I/O) devices, which may bemouse, keyboards, modems, network interfaces, printers, and otherdevices that are well known in the art. Typically, the input/outputdevices 109 are coupled to the system through input/output controllers108. The volatile RAM 105 is typically implemented as dynamic RAM(DRAM), which requires power continuously in order to refresh ormaintain the data in the memory. The non-volatile memory 106 istypically a magnetic hard drive or a magnetic optical drive or anoptical drive or a DVD RAM or other type of memory system, whichmaintains data even after power is removed from the system. Typically,the non-volatile memory will also be a random access memory, althoughthis is not required. While FIG. 1 shows that the non-volatile memory isa local device coupled directly to the rest of the components in thedata processing system, it will be appreciated that the presentinvention may utilize a non-volatile memory which is remote from thesystem, such as a network storage device which is coupled to the dataprocessing system through a network interface such as a modem orEthernet interface. The bus 101 may include one or more buses connectedto each other through various bridges, controllers, and/or adapters, asis well known in the art. In one embodiment, the I/O controller 108includes a USB (Universal Serial Bus) adapter for controlling USBperipherals.

Text Extraction Method and Apparatus from Document Image with ComplexBackground

In the method and apparatus of extracting text from document image withcomplex background according to the present invention, the edgeinformation is used to extract text and in order to overcome thedisturbances of the background edges, it takes the following noveloperations: 1) removing long Connected Component formed by backgroundobjects or very close texts (named long background edge ConnectedComponents) in edge map based on edge classification; 2) recalculatingedge map with feedback of an edge Connected Component size; 3) markingtext map based on edge Connected Component classification.

Here, the Connected Component is the area covering the connected area,and the edge classification means to classify the edge pixels intopositive edge or negative edge, which will be detailed described in thefollowing description. The edge map is image which only includes edgepixels of object, and feed of an edge Connected Component size means thesize of an edge Connected Component can be looked as a kind of feedback,with which the edge map can be determined whether to be recalculated ornot.

The method of extracting text from document image with complexbackground according to the present invention is comprised generally ofthe steps of: (1) calculating edge map; (2) classifying edge into twotypes of “positive edge” and “negative edge” and searching ConnectedComponent (CC) formed by edges of the same type, if a ConnectedComponent formed by edges of the same type is long enough, removing itsedges; (3) searching Connected Components formed by edges of both types(named edge Connected Component), if the size of a Connected Componentformed by edges of both types is too large, recalculating edge map inits bounding box and searching Connected Components again; (4)classifying Connected Components into three types of “normal-text”,“reverse-text” and “background”, and generating a mark map, wherein theforeground pixels of “normal-text” Connected Components are marked as“normal-text”, the foreground pixels of “reverse-text” ConnectedComponents are marked as “reverse-text”, and the rest pixels are markedas “background”; (5) on the mark map, searching Connected Componentsformed by pixels with the same mark (named text Connected Component),and forming the Connected Components into text rows.

In the following, the embodiments of the method of extracting text fromdocument image with complex background and the apparatus of extractingtext from document image with complex background according to thepresent invention will be explained by referring to the accompanyingdrawings. FIG. 2 is a flow chart showing the method of extracting textfrom document image with complex background according to the presentinvention.

As shown in FIG. 2, after inputting the image, in step 1, an edge map iscalculated by performing Sobel operator on the original image, and thenthe gradient image is binarized.

Then, in step 2, the long background edge Connected Components based onedge classification is removed.

In this step, Connected Component analysis is performed on the edge map.For the edge map calculated from an image with complex background, thereare two kinds of Connected Components that will greatly disturb theConnected Component analysis processing, i.e. Connected Componentsformed by edges of long lines and touched Connected Components formed bytext edges which are very close to each other.

The above two kinds of Connected Components may appear in theneighborhood of Connected Components formed by text edges, or even touchwith Connected Components formed by text edges, which will disturb theConnected Component analysis. Therefore, it is desired to find a way toremove them before the Connected Component analysis. For this purpose,an edge classification method is used in the present invention.

According to the method of the present invention, the edges areclassified into negative edges and positive edges according to theirgradient direction. Suppose P_(o) is the gray level of the current edgepixel, p_(neighbor) is the gray level of its 8-neighbors. The type ofthe current edge pixel is determined by the following formula:$\begin{matrix}{{{Edge}\quad{type}} = \{ \begin{matrix}{{{negative}\quad{edge}},} & {{{if}\quad{{P_{o} - {\max( P_{neighbor} )}}}} <} \\\quad & {{P_{o} - {\min( P_{neighbor} )}}} \\{{{positive}\quad{edge}},} & {else}\end{matrix} } & (1)\end{matrix}$

After the edge classification, the negative edge Connected Componentsand the positive edge Connected Components will be searchedindependently. If any of those Connected Components is longer than apredetermined threshold, such as 100, it is deemed as a backgroundConnected Component, and it is removed from the edge map. After thisoperation, edges belong to long lines are removed. Touching edge ofclose texts are also removed, thus the rest edges are separated. FIG. 3shows an example for edge classification and results of removing longbackground edge Connected Components.

It can be seen from the results shown in FIG. 3 that the disturbances ofnot only the long line but also the close texts can be removed accordingto the method of the present invention.

Removing edge Connected Component of long line is easily realized, andthere are many kinds of method, such as disclosed in the above JapanesePatent Application Laid-open No. JP-A-2000-20714. However, in the edgemap, close text row also may form a long Connected Component. Text rowis what desired aims, which should not be simply removed. Afterclassification, in text neighborhood there are two kinds edge ConnectedComponent. One is outer edge Connected Component, and the other is inneredge Connected Component, as shown in FIG. 3. It can be seen that theouter edge Connected Component is long, but inner edge ConnectedComponent is comparatively shorter. If the outer long one is removedfrom the edge map, the rest of inner edge Connected Component also canform text contour, which can be also used to further edge analysis.

After the long Connected Component formed by background objects and veryclose texts is removed from the edge map based on the edgeclassification, as described in the above step 2, the edge map will berecalculated with the feedback of Connected Component size in step 3.

After long background edge Connected Components were removed, theConnected Components formed by edges of both types are searched again(Herein, it does not need to distinguish “negative” and “positive”).

Since the edge-based text extraction method is supposed to be applied ontexts with size smaller than 60 pixels, the Connected Components thatare larger than 60 pixels can be thrown away (Connected Component sizeis often decided by the shorter border of Connected Component's boundingbox). But in image with complex background, the edges of text and thatof background touch with each other, which may form Connected Componentswith size bigger than 60 pixels. So it is needed to deal with thissituation. By adjusting the edge threshold in the bounding box of eachlarge Connected Component, some edge pixels with relatively low contrastcan be eliminated (These edges shall belong to background objects). Theflowchart and an example are illustrated in FIG. 4 and FIG. 5respectively.

FIG. 4 shows the flow chart of recalculating the local edge map with thefeedback of Connected Components size according to the presentinvention.

At first, the Connected Components formed by edges of both types withoutdistinguishing the negative edge and positive edge are searched again onthe input edge map in step S41.

Then, in step S42, the pixel number of the Connected Components iscompared with a predetermined threshold, such as 60 pixels, to decidewhether the edge map near the Connected Components should berecalculated. If the pixel number of the Connected Components is smallerthan 60 pixels, the process is ended and forwards to step 4 of FIG. 2.

On the other hand, if it is decided that the pixel number of theConnected Components is not smaller than the predetermined threshold instep S42, it means that the Connected Components may belong to thebackground and should be thrown away, and the process is forward to stepS43.

In step S43, the threshold value is increased by a predetermined value,such as 20, so as to recalculate the local edge map and remove thedisturbance of the complex background.

Then, in step S44, the corresponding gradient block is binarized againwith the new threshold and the single characters can be separated fromthe complex background.

Thereafter, in step S45 it is judged whether the pixel number of all theConnected Components is smaller than 60 pixels. If yes, the process isforward to step 4 of FIG. 2.

Otherwise, if it is not the pixel number of all the Connected Componentsis smaller than 60 pixels, the process is forward to step S46 to searchanother Connected Components whose pixel number is larger than 60pixels. Then, the process is returned to step S46 and goes on to processsuch Connected Components whose pixel number is larger than 60 pixels.

An example and the result of removing the disturbance of the backgroundnear the text by recalculating the edge map with the feedback ofConnected Components according to the present invention are shown inFIG. 5. In the example of FIG. 5, the edges of the black rectangle ofthe bounding boxes of large Connected Components will be recalculated,and the rectangles with light color are the bounding boxes of smallConnected Components and do not need to be recalculated their edge maps.

As for the above described step 3, proper thresholding method of theconventional prior art is also a substitute, but it is hard to acquireproper threshold for the whole image. With the feedback of edgeConnected Component size, the aim can be focused on the ConnectedComponent covering regions. In these regions, the desired texts can beacquired by using the local information instead of the universalinformation.

In addition, the backgrounds and texts often have different contrast inlocal region. Improper edge threshold will result in edge touching. Byadjusting the edge threshold and recalculating the edge map according tothe above described steps, edge of background objects and that ofdesired texts can be easily separated. Once separated, it is easier toremove background object by analysis of their edge than thresholdingmethod. What's more, by using the recalculated edge map it is easy todecide that texts are normal or reverse, which will benefit the furthertext row merging.

After this step, most edge pixels that belong to the background objectsare removed, and the ones belong to text are reserved. In this way, thesingle characters can be easily separated from the background. It willgreatly help to locate text accurately.

Now returning back to FIG. 2, after the local edge map is recalculatedwith the feed back of edge Connected Component size in step 3, theprocess is forward to step 4.

In step 4, the text map is marked based on the edge Connected Componentclassification. At this step, edge Connected Components gotten in theprevious step are classified into three types of “normal-text”,“reverse-text” and “background”. Then a mark map is generated, with theforeground pixels of “normal-text” Connected Components are marked as“normal-text”, the foreground pixels of “reverse-text” ConnectedComponents are marked as “reverse-text”, and the rest pixels are markedas “background”.

Thus, there are three types of pixels on the mark map: “normal-text”,“reverse-text” and “background”. The mark map will help to mergecharacters with similar properties (namely, “normal-text” or“reverse-text”) into text rows, as well as throw away the non-textcomponents. Further more, the mark map will help to better binarize textrows which will be gotten in the next step 5.

Then, in step 5, the text Connected Component is searched and merged torow. On the mark map, Connected Components formed by pixels with thesame mark (named text Connected Components) are searched and formed intotext rows. There are mainly two reasons for forming texts ConnectedComponents into rows. The first one is that the marked text map cannotbe used as the last binary image, because some text may be missing andmany noises may exist. By forming text Connected Components into rows,it is easy to find some missing texts and remove some noises. The otherreason is that the previous operation of filtering text edge ConnectedComponent is not so sure, because it is difficult to judge whether aConnected Component is a text or not by using only its own features. Butfor text rows, it is much easier because more effective features forclassifying text rows can be found.

There are many text row forming methods in prior arts. The presentinvention adopts one of these methods with the following steps.

Step 51: Finding Connected Components with same mark in the edge map ofthe text image.

Step 52: Merging the intersected Connected Components.

Step 53: Throwing away the non-text Connected Components.

Step 54: Forming row seed by using Close Connected Components, andmerging the other Connected Components to row.

Step 55: Forming row seed by using far Connected Components, and mergingthe Connected Components left by step 54 to row.

Step 56: Forming row seed by using the same Connected Components, andmerging Connected Components left by step 55 to row.

Step 57: Judging each merged row whether it is a real text row or not.

After the above described process, the clear text can be extracted fromthe image with complex background. FIG. 6 and FIG. 7 give the results oftext extraction by using the text extraction method according to thepresent invention.

The text extraction method according to the present invention is mainlyfor extracting texts in image with complex background. It can mainly beused in such fields of the preprocessing in OCR (Optical CharacterRecognition), the text-based video retrieval and the document imagecompression, etc.

A typical application is shown in FIG. 8, in which the edge based textextraction method according to the present invention is firstly used toprocess the color document image. Then, the binarized text row isprocessed with the optical character recognition method and therecognized characters are output.

Next, the apparatus of extracting text from document image with complexbackground according to the present invention will be described byreferring to the accompanying drawings. FIG. 9 is a block diagram of theapparatus of extracting text from document image with complex backgroundaccording to an embodiment of the present invention.

As shown in FIG. 9, the apparatus of extracting text from document imagewith complex background according to an embodiment of the presentinvention comprises an edge map calculation unit 901, a long backgroundConnected Components Remove unit 902, an edge map recalculation unit903, a text map mark unit 904 and a text Connected Component search andmerge unit 905.

The edge map calculation unit 901 calculates the edge map of the inputdocument image and output the calculated edge map into the longbackground Connected Components Remove unit 902.

The long background Connected Components Remove unit 902 classifies theedges in the edge map calculated by the edge map calculation unit 901into two types of “positive edge” and “negative edge” and searches theConnected Component (CC) formed by edges of the same type. If aConnected Component formed by edges of the same type is long enough, thelong background Connected Components Remove unit 902 removes its edges.

The edge map recalculation unit 903 searches Connected Components formedby edges of both types (named edge Connected Component) in the edge mapwith the long Connected Components formed by edges of the same typesbeing removed by the long background Connected Components Remove unit902. If the size of a Connected Component formed by edges of both typesis too large, the edge map recalculation unit 903 recalculates edge mapin its bounding box and searches Connected Components again.

After recalculating the edge map by the edge map recalculation unit 903,the text map mark unit 904 classifies the Connected Components intothree types of “normal-text”, “reverse-text” and “background”, andgenerates a mark map. In the mark map generated by the text map markunit 904, the foreground pixels of “normal-text” Connected Componentsare marked as “normal-text”, the foreground pixels of “reverse-text”Connected Components are marked as “reverse-text”, and the rest pixelsare marked as “background”.

The text Connected Component search and merge unit 905 searches on themark map generated by the text map mark unit 904 the ConnectedComponents formed by pixels with the same mark (named text ConnectedComponent), and forms the Connected Components into text rows.

All the detailed processes performed in the above mentioned edge mapcalculation unit 901, the long background Connected Components Removeunit 902, the edge map recalculation unit 903, the text map mark unit904 and the text Connected Component search and merge unit 905 of theapparatus of extracting text from document image with complex backgroundaccording to the present invention are same as the above steps 1 to 5described by referring to FIG. 2 respectively and thus are omitted here.

In addition, those skilled in the art should be understand that theapparatus of extracting text from document image with complex backgroundaccording to an embodiment of the present invention also should comprisean input unit for inputting the document image and an outputting unitfor outputting the binarized text row after the text being extractedwith the above method and apparatus according to the present invention.

Apparently, those skilled in the art also should be understand that theapparatus of extracting text from document image with complex backgroundaccording to an embodiment of the present invention can be furtherimplemented as comprising a first edge extracting means for extractingedges which have higher contrast than a first contrast threshold fromsaid image; a searching means for searching connected edges from saidextracted edges; a second edge extracting means for extracting edgeswhich have higher contrast than a second contrast threshold in case thatthe pixels number of said searched connected edges is bigger than apredetermined size, wherein said second contrast threshold is higherthan said first contrast threshold.

In a preferred embodiment of the present invention, the second edgeextracting means determines that said searched connected edges is a textedge in case that the pixels number of said searched connected edges issmaller than or equal to said predetermined size.

Furthermore, the second edge extracting means also can extracts edgeswhich have higher contrast than the second contrast threshold only fromsaid connected edges in case that the pixels number of said connectededges is bigger than said predetermined size.

In another preferred embodiment of the present invention, the secondedge extracting means extracts edges which have higher contrast than thesecond contrast threshold from the area covering said connected edges incase that the pixels number of said area is bigger than saidpredetermined size. In such a case, the apparatus of extracting textfrom document image with complex background further comprises a secondconnected edge searching means for searching connected edges from saidedges extracted in said second edge extracting means; a third edgeextracting means for extracting edges which have higher contrast than athird contrast threshold in case that the pixels number of said searchedconnected edge is bigger than said predetermined size, wherein saidthird contrast threshold is higher than said second contrast threshold.

The apparatus of extracting text from document image with complexbackground according to present invention also can comprises an edgeclassifying means for classifying the edges into two types of “positiveedge” and “negative edge” on the basis of the following formula:${{Edge}\quad{type}} = \{ \begin{matrix}{{{negative}\quad{edge}},} & {{{if}\quad{{P_{o} - {\max( P_{neighbor} )}}}} <} \\\quad & {{P_{o} - {\min( P_{neighbor} )}}} \\{{{positive}\quad{edge}},} & {else}\end{matrix} $

where P_(o) is the gray level of the current edge pixel, p_(neighbor) isthe gray level of its N-neighbors; and an edge removing means forremoving an area covering the connected area as a background if thepixels number of said area covering the connected edges formed by thesame type of edge is longer than a predetermined threshold.

Still in another embodiment of the present invention, the apparatus ofextracting text from document image with complex background furthercomprises a text map marking means for marking out the text from theextracted edges, wherein in the foreground pixels of the area coveringthe connected edges are marked as “normal-text”, the foreground pixelsof the area covering the reverse connected edges are marked as“reverse-text”, and the rest pixels are marked as “background”. Theapparatus of extracting text from document image with complex backgroundalso comprises a means for searching and forming the text area formed bypixels with the same mark into text rows.

According to another preferred embodiment of the present invention,another apparatus of extracting text from document image with complexbackground comprises an adjusting means for adjusting the contrastthreshold; a text area determining means for determining where is thetext area based on said adjusted contrast threshold; wherein saidadjusting means comprises a target area determining means for extractingthe edges which have higher contrast than said contrast threshold fromthe target area, searching the connected edges from said extractededges, and determining whether the area covering said searched connectededges should be a new target area; wherein said adjusting means enlargessaid contrast threshold when said determined new target area is biggerthan a predetermined size, and finishes adjustment of said contrastthreshold when said determined new target area is smaller than or equalto the predetermined size; and wherein the text area determining meansdetermines that the target area corresponding to said contrast thresholdwhose adjustment is finished should be the text area.

The apparatus of extracting text from document image with complexbackground according to the above described structure can furthercomprises an edge classifying means for classifying the edges into twotypes of “positive edge” and “negative edge” on the basis of thefollowing formula: ${{Edge}\quad{type}} = \{ \begin{matrix}{{{negative}\quad{edge}},} & {{{if}\quad{{P_{o} - {\max( P_{neighbor} )}}}} <} \\\quad & {{P_{o} - {\min( P_{neighbor} )}}} \\{{{positive}\quad{edge}},} & {else}\end{matrix} $

where P_(o) is the gray level of the current edge pixel, p_(neighbor) isthe gray level of its N-neighbors; and an edge removing means forremoving an area covering the connected area as a background if thepixels number of said area covering the connected edges formed by thesame type of edge is longer than a predetermined threshold.

Also the apparatus of extracting text from document image with complexbackground with the above mentioned structure may further comprises asearching means for searching the area covering the connected edgesformed by both types of edges without distinguishing the negative edgeand positive edge; a local edge recalculating means for, if the pixelsnumber of the searched area covering the connected edges formed by bothtypes of edges without distinguishing the negative edge and positiveedge is larger than a second predetermined threshold, recalculating thelocal edge of the searched area whose pixels number is larger than thesecond predetermined threshold; and a second removing means for removingthe disturbance of the complex background on the basis of therecalculated local edge.

In one preferred embodiment of the above said apparatus of extractingtext from document image with complex background, the local edgerecalculating means increases the binarizing threshold a predeterminedvalue and binarizes the gradient block around the searched area whosepixels number is larger than the second predetermined threshold by usingthe increased binarizing predetermined threshold.

In another embodiment of the present invention, the apparatus ofextracting text from document image with complex background furthercomprises a text map marking means for marking out the text from theextracted edges, wherein in the foreground pixels of the area coveringthe connected edges are marked as “normal-text”, the foreground pixelsof the area covering the reverse connected edges are marked as“reverse-text”, and the rest pixels are marked as “background”.Furthermore, the apparatus of extracting text from document image withcomplex background also may comprises means for searching and formingthe text area formed by pixels with the same mark into text rows.

In addition, the apparatus of extracting text from document image withcomplex background according to an embodiment of the present inventionalso can be implemented as comprising means for removing long ConnectedComponent formed by background objects or very close texts in an edgemap of the document image based on edge classification; means forrecalculating a new edge map of the document image having long ConnectedComponent formed by background objects or very close texts been removedby using feedback of an edge Connected Component size; and means formarking text map based on edge Connected Component classification.

The apparatus of extracting text from document image with complexbackground according to one preferred embodiment of the presentinvention further comprises a means for calculating the edge map of thedocument image by performing Sobel operator on the original image of thedocument image.

In one embodiment of the present invention, the means for removing longConnected Component classifies the edges in the edge map into two typesof “positive edge” and “negative edge” on the basis of the followingformula: ${{Edge}\quad{type}} = \{ \begin{matrix}{{{negative}\quad{edge}},} & {{{if}\quad{{P_{o} - {\max( P_{neighbor} )}}}} <} \\\quad & {{P_{o} - {\min( P_{neighbor} )}}} \\{{{positive}\quad{edge}},} & {else}\end{matrix} $

where P_(o) is the gray level of the current edge pixel, p_(neighbor) isthe gray level of its N-neighbors; and if any of Connected Componentsformed by the same type of edge is longer than a predeterminedthreshold, it is deemed as a background Connected Component and isremoved from the edge map.

Preferably, N is equal to 8 and the predetermined threshold is equal to100.

In another embodiment of the present invention, the means forrecalculating the new edge map searches the Connected Components formedby edges of both types without distinguishing the negative edge andpositive edge; if the searched Connected Components formed by edges ofboth types without distinguishing the negative edge and positive edge islarger than a second predetermined threshold, recalculates the localedge map of the searched Connected Components larger than the secondpredetermined threshold; and removes the disturbance of the complexbackground on the recalculated local edge map. Preferably, the secondpredetermined threshold is equal to 60.

According to another preferable embodiment of the present invention, themeans for marking text map classifies the edge Connected Components intothree types of “normal-text”, “reverse-text” and “background”, and thusa mark map is generated, wherein the foreground pixels of “normal-text”Connected Components are marked as “normal-text”, the foreground pixelsof “reverse-text” Connected Components are marked as “reverse-text”, andthe rest pixels are marked as “background”.

The apparatus of extracting text from document image with complexbackground having the above mentioned structure further comprises ameans for searching text Connected Components formed by pixels with thesame mark on the mark map and forming the text Connected Components intotext rows.

In one preferred apparatus of extracting text from document image withcomplex background of the present invention, the means for searching andforming the text Connected Components d1) finding Connected Componentswith same mark in the edge map of the text image; d2) merging theintersected Connected Components; d3) throwing away the non-textConnected Components; d4) forming row seed by using Close ConnectedComponents, and merging the other Connected Components to row; d5)forming row seed by using far Connected Components, and merging the leftConnected Components to row; d6) forming row seed by using the sameConnected Components, and merging the left Connected Components to row;and d7) judging each merged row whether it is a real text row or not.

Besides the above mentioned concrete embodiments of the presentinvention's method and apparatus, the objects of the invention may alsobe realized through running a program or a set of programs on anyinformation processing equipment as described above, which may becommunicated with any subsequent processing apparatus. Said informationprocessing equipment and subsequent processing apparatus may be allwell-known universal equipments.

Therefore, it is important to note that the present invention includes acase wherein the invention is achieved by directly or remotely supplyinga program (a program corresponding to the illustrated flow chart in theembodiment) of software that implements the functions of theaforementioned embodiments to a system or apparatus, and reading out andexecuting the supplied program code by a computer of that system orapparatus. In such case, the form is not limited to a program as long asthe program function can be provided.

Therefore, the program code itself installed in a computer to implementthe functional process of the present invention using computerimplements the present invention. That is, the present inventionincludes the computer program itself for implementing the functionalprocess of the present invention.

In this case, the form of program is not particularly limited, and anobject code, a program to be executed by an interpreter, script data tobe supplied to an OS, and the like may be used as along as they have theprogram function.

As a recording medium for supplying the program, for example, a floppydisk, hard disk, optical disk, magneto optical disk, MO, CD-ROM, CD-R,CD-RW, magnetic tape, nonvolatile memory card, ROM, DVD (DVD-ROM,DVD-R), and the like may be used.

As another program supply method, connection may be established to agiven home page on the Internet using a browser on a client computer,and the computer program itself of the present invention or a file,which is compressed and includes an automatic installation function, maybe downloaded from that home page to a recording medium such as a harddisk or the like, thus supplying the program. Also, program codes thatform the program of the present invention may be broken up into aplurality of files, and these files may be downloaded from differenthome pages. That is, the present invention also includes a WNW serverthat makes a plurality of users download program files for implementingthe functional process of the present invention using a computer.

Also, a storage medium such as a CD-ROM or the like, which stores theencrypted program of the present invention, may be delivered to theuser, the user who has cleared a predetermined condition may be allowedto download key information that decrypts the program from a home pagevia the Internet, and the encrypted program may be executed using thatkey information to be installed on a computer, thus implementing thepresent invention.

The functions of the aforementioned embodiments may be implemented notonly by executing the readout program code by the computer but also bysome or all of actual processing operations executed by an OS or thelike running on the computer on the basis of an instruction of thatprogram.

Furthermore, the functions of the aforementioned embodiments may beimplemented by some or all of actual processes executed by a CPU or thelike arranged in a function extension board or a function extensionunit, which is inserted in or connected to the computer, after theprogram read out from the recording medium is written in a memory of theextension board or unit.

What has been describes herein is merely illustrative of the applicationof the principles of the present invention. For example, the functionsdescribed above as implemented as the best mode for operating thepresent invention are for illustration purposes only. As a particularexample, for instance, other design may be used for obtaining andanalyzing waveform data to determine speech. Also, the present inventionmay be used for other purposes besides detecting speech. Accordingly,other arrangements and methods may be implemented by those skilled inthe art without departing from the scope and spirit of this invention.

1. A method of extracting text from document image with complexbackground comprising the steps of: a first edge extracting step ofextracting edges which have higher contrast than a first contrastthreshold from said image; a searching step of searching connected edgesfrom said extracted edges; a second edge extracting step of extractingedges which have higher contrast than a second contrast threshold incase that the pixels number of said searched connected edges is biggerthan a predetermined size, wherein said second contrast threshold ishigher than said first contrast threshold.
 2. The method of extractingtext from document image with complex background according to claim 1,wherein the second edge extracting step determines that said searchedconnected edges is a text edge in case that the pixels number of saidsearched connected edges is smaller than or equal to said predeterminedsize.
 3. The method of extracting text from document image with complexbackground according to claim 1, wherein the second edge extracting stepextracts edges which have higher contrast than the second contrastthreshold only from said connected edges in case that the pixels numberof said searched connected edges is bigger than said predetermined size.4. The method of extracting text from document image with complexbackground according to claim 1, wherein the second edge extracting stepextracts edges which have higher contrast than the second contrastthreshold from the area covering said connected edges in case that thepixels number of said area is bigger than said predetermined size. 5.The method of extracting text from document image with complexbackground according to claim 2, further comprising the steps of: asecond connected edge searching step of searching connected edges fromsaid edges extracted in said second edge extracting step; a third edgeextracting step of extracting edges which have higher contrast than athird contrast threshold in case that the pixels number of said searchedconnected edge is bigger than said predetermined size, wherein saidthird contrast threshold is higher than said second contrast threshold.6. The method of extracting text from document image with complexbackground according to claim 1, further comprising the steps of: anedge classifying step of classifying the edges into two types of“positive edge” and “negative edge” on the basis of the followingformula: ${{Edge}\quad{type}} = \{ \begin{matrix}{{{negative}\quad{edge}},} & {{{if}\quad{{P_{o} - {\max( P_{neighbor} )}}}} <} \\\quad & {{P_{o} - {\min( P_{neighbor} )}}} \\{{{positive}\quad{edge}},} & {else}\end{matrix} $ where P_(o) is the gray level of the current edgepixel, p_(neighbor) is the gray level of its N-neighbors; and an edgeremoving step of removing an area covering the connected area as abackground if the pixels number of said area covering the connectededges formed by the same type of edge is longer than a predeterminedthreshold.
 7. The method of extracting text from document image withcomplex background according to claim 1, further comprising a text mapmarking step of marking out the text from the extracted edges, whereinin the foreground pixels of the area covering the connected edges aremarked as “normal-text”, the foreground pixels of the area covering thereverse connected edges are marked as “reverse-text”, and the restpixels are marked as “background”.
 8. The method of extracting text fromdocument image with complex background according to claim 7, furthercomprising the step of searching and forming the text area formed bypixels with the same mark into text rows.
 9. A method of extracting textfrom document image with complex background comprising the steps of: anadjusting step of adjusting the contrast threshold; a text areadetermining step of determining where is the text area based on saidadjusted contrast threshold; wherein said adjusting step comprises astep of target area determining step of extracting the edges which havehigher contrast than said contrast threshold from the target area,searching the connected edges from said extracted edges, and determiningwhether the area covering said searched connected edges should be a newtarget area; wherein said adjusting step enlarges said contrastthreshold when said determined new target area is bigger than apredetermined size, and finishes adjustment of said contrast thresholdwhen said determined new target area is smaller than or equal to thepredetermined size; and wherein the text area determining stepdetermines that the target area corresponding to said contrast thresholdwhose adjustment is finished should be the text area.
 10. The method ofextracting text from document image with complex background according toclaim 9, further comprising the steps of: an edge classifying step ofclassifying the edges into two types of “positive edge” and “negativeedge” on the basis of the following formula:${{Edge}\quad{type}} = \{ \begin{matrix}{{{negative}\quad{edge}},} & {{{if}\quad{{P_{o} - {\max( P_{neighbor} )}}}} <} \\\quad & {{P_{o} - {\min( P_{neighbor} )}}} \\{{{positive}\quad{edge}},} & {else}\end{matrix} $ where P_(o) is the gray level of the current edgepixel, p_(neighbor) is the gray level of its N-neighbors; and an edgeremoving step of removing an area covering the connected area as abackground if the pixels number of said area covering the connectededges formed by the same type of edge is longer than a predeterminedthreshold.
 11. The method of extracting text from document image withcomplex background according to claim 10, further comprising the stepsof: searching the area covering the connected edges formed by both typesof edges without distinguishing the negative edge and positive edge; ifthe pixels number of the searched area covering the connected edgesformed by both types of edges without distinguishing the negative edgeand positive edge is larger than a second predetermined threshold,recalculating the local edge of the searched area whose pixels number islarger than the second predetermined threshold; and removing thedisturbance of the complex background on the basis of the recalculatedlocal edge.
 12. The method of extracting text from document image withcomplex background according to claim 11, wherein the step ofrecalculating the local edge comprising the steps of: increasing thebinarizing threshold a predetermined value; and binarizing the gradientblock around the searched area whose pixels number is larger than thesecond predetermined threshold by using the increased binarizingpredetermined threshold.
 13. The method of extracting text from documentimage with complex background according to claim 9, further comprises atext map marking step of marking out the text from the extracted edges,wherein in the foreground pixels of the area covering the connectededges are marked as “normal-text”, the foreground pixels of the areacovering the reverse connected edges are marked as “reverse-text”, andthe rest pixels are marked as “background”.
 14. The method of extractingtext from document image with complex background according to claim 13,further comprising the step of searching and forming the text areaformed by pixels with the same mark into text rows.
 15. An apparatus ofextracting text from document image with complex background, comprising:a first edge extracting means for extracting edges which have highercontrast than a first contrast threshold from said image; a searchingmeans for searching connected edges from said extracted edges; a secondedge extracting means for extracting edges which have higher contrastthan a second contrast threshold in case that the pixels number of saidsearched connected edges is bigger than a predetermined size, whereinsaid second contrast threshold is higher than said first contrastthreshold.
 16. The apparatus of extracting text from document image withcomplex background according to claim 15, wherein the second edgeextracting means determines that said searched connected edges is a textedge in case that the pixels number of said searched connected edges issmaller than or equal to said predetermined size.
 17. The apparatus ofextracting text from document image with complex background according toclaim 15, wherein the second edge extracting means extracts edges whichhave higher contrast than the second contrast threshold only from saidconnected edges in case that the pixels number of said connected edgesis bigger than said predetermined size.
 18. The apparatus of extractingtext from document image with complex background according to claim 15,wherein the second edge extracting means extracts edges which havehigher contrast than the second contrast threshold from the areacovering said connected edges in case that the pixels number of saidarea is bigger than said predetermined size.
 19. The apparatus ofextracting text from document image with complex background according toclaim 16, further comprising: a second connected edge searching meansfor searching connected edges from said edges extracted in said secondedge extracting means; a third edge extracting means for extractingedges which have higher contrast than a third contrast threshold in casethat the pixels number of said searched connected edge is bigger thansaid predetermined size, wherein said third contrast threshold is higherthan said second contrast threshold.
 20. The apparatus of extractingtext from document image with complex background according to claim 15,further comprising: an edge classifying means for classifying the edgesinto two types of “positive edge” and “negative edge” on the basis ofthe following formula: ${{Edge}\quad{type}} = \{ \begin{matrix}{{{negative}\quad{edge}},} & {{{if}\quad{{P_{o} - {\max( P_{neighbor} )}}}} <} \\\quad & {{P_{o} - {\min( P_{neighbor} )}}} \\{{{positive}\quad{edge}},} & {else}\end{matrix} $ where P_(o) is the gray level of the current edgepixel, p_(neighbor) is the gray level of its N-neighbors; and an edgeremoving means for removing an area covering the connected area as abackground if the pixels number of said area covering the connectededges formed by the same type of edge is longer than a predeterminedthreshold.
 21. The apparatus of extracting text from document image withcomplex background according to claim 15, further comprising a text mapmarking means for marking out the text from the extracted edges, whereinin the foreground pixels of the area covering the connected edges aremarked as “normal-text”, the foreground pixels of the area covering thereverse connected edges are marked as “reverse-text”, and the restpixels are marked as “background”.
 22. The apparatus of extracting textfrom document image with complex background according to claim 21,further comprising a means for searching and forming the text areaformed by pixels with the same mark into text rows.
 23. An apparatus ofextracting text from document image with complex background comprising:an adjusting means for adjusting the contrast threshold; a text areadetermining means for determining where is the text area based on saidadjusted contrast threshold; wherein said adjusting means comprises atarget area determining means for extracting the edges which have highercontrast than said contrast threshold from the target area, searchingthe connected edges from said extracted edges, and determining whetherthe area covering said searched connected edges should be a new targetarea; wherein said adjusting means enlarges said contrast threshold whensaid determined new target area is bigger than a predetermined size, andfinishes adjustment of said contrast threshold when said determined newtarget area is smaller than or equal to the predetermined size; andwherein the text area determining means determines that the target areacorresponding to said contrast threshold whose adjustment is finishedshould be the text area.
 24. The apparatus of extracting text fromdocument image with complex background according to claim 23, furthercomprising: an edge classifying means for classifying the edges into twotypes of “positive edge” and “negative edge” on the basis of thefollowing formula: ${{Edge}\quad{type}} = \{ \begin{matrix}{{{negative}\quad{edge}},} & {{{if}\quad{{P_{o} - {\max( P_{neighbor} )}}}} <} \\\quad & {{P_{o} - {\min( P_{neighbor} )}}} \\{{{positive}\quad{edge}},} & {else}\end{matrix} $ where P_(o) is the gray level of the current edgepixel, p_(neighbor) is the gray level of its N-neighbors; and an edgeremoving means for removing an area covering the connected area as abackground if the pixels number of said area covering the connectededges formed by the same type of edge is longer than a predeterminedthreshold.
 25. The apparatus of extracting text from document image withcomplex background according to claim 24, further comprising: asearching means for searching the area covering the connected edgesformed by both types of edges without distinguishing the negative edgeand positive edge; a local edge recalculating means for, if the pixelsnumber of the searched area covering the connected edges formed by bothtypes of edges without distinguishing the negative edge and positiveedge is larger than a second predetermined threshold, recalculating thelocal edge of the searched area whose pixels number is larger than thesecond predetermined threshold; and a second removing means for removingthe disturbance of the complex background on the basis of therecalculated local edge.
 26. The apparatus of extracting text fromdocument image with complex background according to claim 25, whereinthe local edge recalculating means increasing the binarizing threshold apredetermined value; and binarizing the gradient block around thesearched area whose pixels number is larger than the secondpredetermined threshold by using the increased binarizing predeterminedthreshold.
 27. The apparatus of extracting text from document image withcomplex background according to claim 23, further comprises a text mapmarking means for marking out the text from the extracted edges, whereinin the foreground pixels of the area covering the connected edges aremarked as “normal-text”, the foreground pixels of the area covering thereverse connected edges are marked as “reverse-text”, and the restpixels are marked as “background”.
 28. The apparatus of extracting textfrom document image with complex background according to claim 27,further comprising means for searching and forming the text area formedby pixels with the same mark into text rows.
 29. An apparatus ofextracting text from document image with complex background comprising:an edge map calculation unit (901) for calculating the edge map of thedocument image; a long background connected edges remove unit (902) forclassifying the edges in the edge map calculated by the edge mapcalculation unit (901) into two types of “positive edge” and “negativeedge”, searching the connected edges formed by edges of the same type,and removing the connected edges formed by edges of the same type longerthan a predetermined threshold; an edge map recalculation unit (903) forsearching connected edges formed by edges of both types in the edge mapwith the long connected edges formed by edges of the same types beingremoved by the long background connected edges remove unit (902), andrecalculating local edge map for the bounding box of a connected edgeformed by edges of both types larger than a second predeterminedthreshold; a text map mark unit (904) for classifying the connectededges into three types of “normal-text”, “reverse-text” and “background”and generating a mark map, wherein the foreground pixels of“normal-text” connected edges are marked as “normal-text”, theforeground pixels of “reverse-text” connected edges are marked as“reverse-text”, and the rest pixels are marked as “background”; and atext connected edge search and merge unit (905) for searching on themark map generated by the text map mark unit (904) the connected edgesformed by pixels with the same mark and forming the connected edges intotext rows.
 30. The apparatus of extracting text from document image withcomplex background as set forth in claim 29, further comprising: aninput unit for inputting the document image; and an outputting unit foroutputting the binarized text row after the text being extracted.
 31. Acomputer program, when executed by a computer, enables the computer toperform acts as claimed in any one of claims 1 to
 8. 32. A computerprogram product in at least one computer-readable medium comprisingfunctional descriptive material that, when executed by a computer,enables the computer to perform acts as claimed in any one of claims 1to
 8. 33. A computer program, when executed by a computer, enables thecomputer to perform acts as claimed in any one of claims 9 to
 14. 34. Acomputer program product in at least one computer-readable mediumcomprising functional descriptive material that, when executed by acomputer, enables the computer to perform acts as claimed in any one ofclaims 9 to 14.